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ABSTRACT 


Helmet-mounted displays of infrared imagery (forward-looking infrared (FLIR)) a low 
helicopter pilots to perform low-level missions at night and in low visibility. However, pilots 
experience high visual and cognitive workload during these missions, and their performance capa- 
bilities may be reduced. Human factors problems inherent in existing systems stem from three 
primary sources: (1) the nature of thermal imagery, (2) the character! sties of specific FLIR sys- 
tem^, and (3) the difficulty of using a FLIR system for flying and/or visually acquiring and 
tracking objects in the environment. The pilot night vision system (PNVS) in the Apache AH-64 
provides a monochrome, 30° by 40° helmet-mounted display of infrared imagery. Thermal 
imagery is inferior to television imagery in both resolution and contrast ratio. Gray shades repre- 
sent temperatures differences rather than brightness variability, and images undergo significant 
changes over time. The limited field of view, displacement of the sensor from the pilot s eye 
position, and monocular presentation of a bright FLIR image (while the other eye remains dar 
adapted) are all potential sources of disorientation, limitations in depth and distance estimation 
sensations of apparent motion, and difficulties in target and obstacle detection. Insufficient infor- 
mation about human perceptual and performance limitations restrains the ability of human factors 
specialists to provide significantly improved specifications, training programs, or alternative 
designs. Additional research is required to determine the most critical problem areas and to pro- 
pose solutions that consider the human as well as the development of technology. 


INTRODUCTION 


In most civil and military operations, helicopter pilots rely on visual cues to maintain situa- 
tional awareness (e.g., estimate the orientation, altitude, speed, and direction of their vehicle; the 
location of hazards in the environment; and their geographical location). Maintaining visual contact 
with the environment is particularly important (and difficult) in nap-of-the-earth (NOE) flight, 
where pilots fly at altitudes between 10 and 30 ft, navigating in and among trees, hills, and build- 
ings. During NOE flight, pilots must keep their eyes "out of the cockpit," rather than focused on 
displays within the cockpit. There is little margin for error. Existing electronic display systems 
do not provide adequately detailed information for visual flightpath control, and guidance algo- 
rithms do not yet exist for automatic NOE flight. 
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At night and in low visibility, the problem is more severe. Sufficient visual information 
about the environment is not available for pilots to navigate safely or identify relevant objects. For 
this reason, light-intensifying goggles and helmet-mounted displays of infrared imagery have been 
developed. This paper will focus on the unique visual environment created by the latter, as helmet- 
mounted displays of infrared imagery (alone or in combination with other sources of visual infor- 
mation) are integral to the design of many advanced helicopters. 

Forward-looking infrared (FLIR) systems provide pilots with a monochromatic video 
unage of the outside scene constructed from thermal differences among environmental features. 
Computer-generated flight symbology may be superimposed on the helmet-mounted display of 
FLIR imagery. Current FLIR pilot night vision systems (PNVS) can be used at night, in total 
darkness, or during the day, to allow pilots to "see" through blowing dust, smog, smoke, or con- 
cealing foliage. 

The FLIR systems used in the Cobra AH- IS and the Apache AH-64 are turret-mounted on 
the nose of the helicopter. Their movement is slaved to the position of the pilot’s helmet, allowing 
die pilot to move the 30° (vertical) by 40° (horizontal) instantaneous field of view (FOV) through a 
"field of regard" of ±90° in azimuth and 65° in elevation (from +20° to -45°) (fig. 1). The infrared 
sensor consists of an array of 180 detectors which provides 360 lines of resolution. This informa- 
tion is transformed into a 87 5-line video image which is displayed on a 1 .92-cm combining lens (a 
monocle) mounted on the helmet immediately in front of the pilot’s right eye. (fig. 2) 


Given the integral role such systems are playing in advanced rotorcraft, it is surprising how 
little is known about human factors problems which are related to the use of these complex and 
highly demanding systems. The problems may be divided into three categories; (1) the unique 
nature of infrared images, (2) specific characteristics of the PNVS, and (3) problems related to the 
task of flying a helicopter at low altitudes in low-visibility conditions. This paper will focus on the 
most critical problem areas and evaluate their effects on pilot perception and performance. 


CHARACTERISTICS OF THERMAL IMAGES 


Thermal images are a visible representation of radiation in the infrared band (8-14 pm in 
the PNVS). Thermal radiation is detected by an array of 1 80 detectors, in current-technology sys- 
tems, which can create a visual display with approximately 360 lines of horizontal resolution. The 
output of each detector is preamplified, entered into a scan converter, transformed into a video 
image, and displayed on a combining lens mounted on the pilot's helmet. 

The temperature of an object depends on the properties of its component materials and on 
its exposure to natural or artificial sources of heat. Its "thermal signature" depends primarily on its 
heat-emitting characteristics. The quality of a thermal image depends on the thermal signatures of 
terrain features and objects; the presence of thermal variability in the environment and atmospheric 
conditions (e.g., ambient temperatures, moisture, dust, and haze); and the sensitivity and size of 
the detectors. Current systems have a limited bandwidth which acts as a low-pass filter, effectively 
limiting the detail with which objects can be depicted. 

Since FLIR images are transformed into video images and displayed on a cathode ray tube 
(CRT), they inherently suffer from all of the shortcomings of video imagery (e.g., limited 
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resolution, restricted contrast sensitivity, and dynamic brightness range). In addition, they are 
displayed monochromatically and provide a two-dimensional representation of the three- 
dimensional world. In comparison to video images, the display provided by the PNVS 1S " so 
subject to the specific properties of FLIR technology and the unique characteristics of the thermal 
(as compared to the visual) properties of objects in the environment. Figure 3 depicts an example 
of a FLIR image with superimposed symbology. 

The meaning of "bright" and "dark" in the thermal image is not necessarily equivalent to 
light and shade in the optical sense. An object may emit little heat because it is shaded, or for a ^ 
variety of other reasons related to the nature of the material and its "thermal history (Lloyd, l 975 )- 
Thus in a given image, there may be "shades" which are partly equivalent to real optical shades, or 
there’may be no shading whatsoever. The human eye has been trained to interpretdark spots as 
shaded areas. These are usually perceived as low spots or valleys in the terrain. Thus, pilots may 
try (inappropriately) to impose the same perceptual rules on thermal images. Furthermore, the 
brightness of a displayed object does not provide accurate range information because objects which 
emit high thermal energy may appear to be closer than they really are. Such misinterpretations of 
the terrain structure may have severe consequences for helicopter flight at very low altitudes. 


The relative temperature of an object changes because of ambient temperature, internal heat 
production, and its heat-emitting characteristics. Thus, its infrared signature may change dynami- 
cally over time. Further, when the temperature of the "foreground" and background are near the 
same value (e.g., the "crossover" point) an object may disappear from the visual display. For 
example, a truck on a snow-covered field would be quite visible while its engine is running, ut 
virtually invisible after sitting with its engine off for several hours. There are relatively predictable 
periods during each day when the temperatures of specific substances are very nearly equal. For 
example, water and vegetation may have two crossover points each day, under some conditions 
(fig. 4). When crossover occurs, the ability of a FLIR system to discriminate is severely 
degraded. The net result is very poor image quality (Berry et al., 1984). 


During the day or soon after sunset, there may be high thermal contrasts, depending on the 
terrain and on atmospheric conditions. When this occurs, there are wide temperature gradients, 
which generate clear and highly detailed images. Later in the night, thermal contrasts gradually 
diminish and images become less detailed. In addition, the effect of solar theimal radiation on the 
temperatures of different substances varies and elements of terrain features may cool at different 

rates during the night. For example, leaves cool more rapidly than branches. Thus, late at nig t 

trees may look as if they have shed their leaves because their temperature approaches that of the 
ambient air temperature. It may be quite confusing for a pilot to pass a grove of fully-leaved trees 
on the way to a mission and a grove of apparently dormant trees on the way back. 


On the other hand, because of the chemical processes, leaves may emit their own heat. 
Thus, when the polarity of the system is set so that dark shades represent cooler objects, leaves are 
very bright in contrast to their dark appearance in optical images. These "blonde" trees seem to 
merge into the background, making it difficult for pilots to spot them from a distance. Such 
dynamic changes require pilots to use complex rules of thumb to interpret visual images, yet accu- 
rate evaluations are critical for pilots flying below treetop level. 


Urban areas generate and accumulate considerable heat during the day, but, as they cool 
during the night, temperatures tend to equalize. This can make it virtually impossible for a pilot to 
identify a specific object (such as a high building) which would stand out in an optical image. 
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Human-made sources of thermal radiation, such as engines, fires, and friction, provide 
small, but significant, sources of infrared radiation. An operating truck, for example, might have a 
hot spot near the location of the engine and another near the wheels. Thus, the thermal "signature" 
of the truck is quite different from its optical image. Furthermore, if the truck remains stationary, 
with its engine off, it may become difficult to discriminate from the surrounding terrain. The 
changing visual appearance of human-made objects presents a particularly critical problem for mil- 
itary pilots performing target identification and tracking. 

Because infrared detectors are sensitive to relative rather than to absolute temperatures, and 
because most FLIR sensors scan horizontally (parallel to the horizon), the horizon may blend with 
the ground and sky (Bohm, 1985). The absence of a clear horizon line may have a detrimental 
effect on spatial orientation and altitude estimation. 


Display Polarity 

Pilots may elect to assign either light or dark values to "hot" objects in the environment. 
Depending on the circumstances, they may alternate between the two polarities, selecting the one 
that provides the clearest image. Unlike the difficulties that people encounter in interpreting nega- 
tives of optical images, pilots can often improve their ability to recognize objects and interpret ter- 
rain features by switching the polarity of the FLIR display. For example, the sky is usually per- 
ceived as a bright area in an optical image, and it is always colder than the terrain. Thus, when 
the polarity is set to white-cold, the sky will appear to be bright. However, this will coincidentally 
result in some shaded areas also appearing as bright areas, in contrast to everyday experience. 
Thus, under a specific set of circumstances, one polarity might provide the most interpretable 
image for targeting or geographical orientation, while the other might be optimal for pilotage. 


Gain and Level 

The visual display may, at any given moment, present only a sample of the dynamic tem- 
perature range. "Gain" and "level" controls allow the pilot to select the desired range of displayed 
temperatures. A specific combination of gain and level may or may not be optimal for a particular 
task. For example, if gain and level are set to be very sensitive to temperature variations within hot 
target areas, an insufficient number of gray shades might be available to provide a detailed image of 
the general scene. Some advanced systems offer automatic control over gain and/or level, to pro- 
vide an optimal presentation of the average range of temperatures, without requiring the pilot to 
make control adjustments. This solution, while intended to reduce pilot workload, may be subop- 
timal for detecting a specific object in a given setting. 

In summary, thermal images have some unique characteristics that result from the nature of 
infrared radiation. Human perceptual skills, which provide efficient tools for interpreting the 
optical world, may be misleading when applied to thermal images. Research is necessary to 
(1) determine how the unique characteristics of infrared imagery interact with various aspects of 
human performance, (2) define the skills that are necessary to use FLIR displays of thermal 
images, and (3) establish how such skills should be acquired. 
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SPECIFIC CHARACTERISTICS OF THE PNVS 


In addition to the inherent characteristics of infrared imagery, many of the human factors 
problems identified in current systems are related to specific components and design limitations o 

the PNVS itself. 


Sensor Location 

In the Apache, the FLIR sensor is mounted 3.5 m in front and 1.2 m below the pilot's eye 
position, creating a displaced eyepoint (fig. 5). Thus, objects within the field of regard of the 
sensor may be physically closer to the sensor than they are to the pilots natural visual reference 
(his eyes) (Berry et al., 1984). During training, pilots must learn to adapt to a different visual ref- 
erence point and adopt slightly different rules of thumb for estimating range and altitude using the 
PNVS display. In addition, objects abeam the sensor (which are no longer visible on the monocle) 
might not have passed the pilot's natural visual reference point, creating the possibility of con u- 
sion if the object is also visible to the pilots' unaided eye (fig. 6). 

Since the sensor is located closer to the ground than are the pilots' eyes, available visual 
motion cues indicate slightly higher apparent velocities than pilots would estimate with direct 
vision. Again, during training, they must learn new rules of thumb to estimate their speed using 
the PNVS display. The displaced eyepoint creates motion parallax problems which are particularly 
severe when large viewing azimuths are encountered. 


Sensor Movement 

In the Apache, the FLIR sensor responds to pilot head movements, moving at a rate of 
approximately 150°/sec. However, the slight delay between movement of the helmet and move- 
ment of the sensor can contribute to motion parallax problems. Although pilots learn to limit the 
frequency and velocity of their head movements to reduce such problems, certain tasks may require 
both rapid and frequent changes in the orientation of the sensor to a specific location or object 
within the FOV of the sensor. 


Helmet-Mounted Display Unit 

In the Honeywell Integrated Helmet and Display Sighting System (IHADSS) used in the 
Apache and the Cobra "surrogate trainer" (where some pilots are familiarized with the system), 
infrared imagery is displayed as a rectangular area on a combiner lens incorporated into the helmet- 
mounted display unit (HDU). The lens is a semitransparent viewing screen that filters light in the 
red and blue range and reflects the composite video image presented in the green wavelength. 1 he 
back of the lens is chemically coated to reduce glare, transmitting 50% of the light incident upon it. 
The lens reflects 80% of the green light rays that exit the HDU toward the pilot's eye. The end 
result of the filtering, magnifying, collimating, and reflecting processes is a two-dimensional, 
monochromatic, monocular display with a maximum of 125-150 ft-L of brightness (Berry et al., 

1984). 
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Field of View 


The image presented to the pilot by the PNVS/IHADSS represents a rectangular FOV of 
30° by 40°. The pilot views an image which is equivalent to a 7-ft television screen viewed from a 
distance of 10 ft (Berry et al., 1984). This relatively narrow FOV eliminates peripheral informa- 
tion that is critical for visual flightpath control. In visual flight, pilots depend on peripheral motion 
cues to estimate speed and orientation and to develop a sense of object’s structure from visual 
motion cues. In addition, pilots must maintain their awareness of significant terrain features, the 
position and identity of stationary objects, and the projected course of moving vehicles that sur- 
round them for navigation, tactical decision-making, and obstacle avoidance. However, the field 
of regard of the sensor limits pilots' abilities to maintain visual contact with objects that are located 
beside or behind their vehicle. 

Surprisingly, little empirical information is available about pilots' FOV requirements for 
pilotage, navigation, and target acquisition or their performance capabilities with different FOV. 
Furthermore, the FOV requirements for a helmet-mounted PNVS are even less well-known. A 
pilot may be faced with the requirement to fly the vehicle while visually tracking a target moving 
off-axis to the direction of flight using the same helmet-mounted display as the primary source of 
visual information for both tasks. 

Considerable effort is being devoted to providing a wider FOV in more advanced systems 
(up to 60° or 90°) or providing different sensitivity for the foveal and peripheral elements of such a 
display. However, it is not clear whether the additional cost will be justified by an improvement in 
performance. Even a 90° FOV does not provide all of the peripheral cues available to the unaided 
eye in good visibility. Furthermore, if the FOV is increased without also improving the resolution 
of the display, the result may be a wide, but inadequately resolved, view of the terrain. 


Display Resolution 

Pilots have identified display resolution as one of the most critical problems in existing 
systems (Bennett and Hart, 1987), although the IHADSS provides 875 lines of display resolution. 
To some extent, the appearance of inadequate display resolution could reflect the fact that the image 
is presented in close proximity to the pilot's eye. For example, the panel-mounted PNVS display 
has the same resolution as the helmet-mounted version, but it is viewed from a greater distance. 
This creates the impression of better resolution. 

In fact, the apparent limitations in display resolution reflect the capabilities of the entire 
system, rather than the quality of the display alone. The effective resolution of the PNVS is less 
than 360 horizontal scan lines. Thus, in a 30° vertical FOV each scan line covers 5-6 min of visual 
angle, as compared to the resolving power of the human eye of about 1 min of arc. This is a sub- 
stantial limitation in the level of detail that is available for presentation by the display system. For 
example, pilots report having great difficulty in detecting wires or other small targets, unless their 
thermal contrast with the surrounding environment is very high. 
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Display Contrast 

Advanced infrared detectors are capable of detecting temperature differences of approxi- 
mately 0 3°C (Haidn, 1985). And a high-quality CRT can display at least 64 shades of gray. 
However the PNVS provides only 10 shades of gray (ranging from bright to dark) to represent 
temperature differences in the environment (Tucker, 1984). This limitation severely restricts the 
level of detail that can be displayed at any one time and may interact with other limitations (e.g., 
limited resolution) to produce an unacceptable image quality. 

Furthermore, specific gain and level selections, which are intended to enhance contrast in 
one region of the total range, might limit detail in another. For example, if the system is set to 
provide maximum contrast between the extremes, discriminations in the midrange will be limited. 
Conversely, when the display is optimized to provide fine discriminations in the midrange, extreme 
thermal signatures may not be discriminable. Because of the restricted number of gray shades 
provided to depict an image, the tolerance for inappropriate gain and level settings is very limited. 


Monocular Presentation 


At night, the image presented by the PNVS/IHADSS effectively limits peripheral vision in 
the right eye, because the display is so bright in comparison to the environment. However a full 
monocular FOV is still available to the unaided left eye (although visible cues may be limited on a 
dark night). Certain details and distance judgments may be obtained more accurately with the 
unaided (left) eye than with the aided (right) eye. Thus, pilots must rely on both sources of visual 
information. However, under most circumstances, the same object viewed by both eyes cannot be 
merged into a coherent binocular image, because of the differences in brightness, perceived size, 
and perceived location (resulting from the displaced eyepoint of the se " s ^;) ^ tter t s 

worse, the right eye may be adapted to the bright image provided by the PNVS/IHADSS system, 
while the left eye might be dark-adapted to the environment. The problem of motion parallax cre- 
ated by the displaced eyepoint provided by the sensor location is particularly great in good visibility 
(where the unaided eye receives a clear image). 


In practice, the use of available visual cues to augment information provided by the sensor 
may create more of a handicap than a help, because of competition between images presented to the 
two eyes (binocular rivalry). One consequence of binocular rivalry is that the information available 
in one eye, by competing for pilot's visual attention, may partially or completely suppress infor- 
mation available to the other eye. Furthermore, since pilots are trained to use both eyes when y- 
ing with a PNVS, they must learn how to process disparate visual cues, or shift their attention 
between their right eye (to use the PNVS) and left eye (to view the terrain or panel instruments.) 

To some extent, the focus of visual attention is under the pilot’s conscious control. However, 
pilots report increasing difficulty in controlling the focus of visual attention as missions progress. 
After less than 1 hr of continuous use, some pilots report they must close one eye (to restore the 
visibility of information in the other eye) or exert significant attentional effort (Bennett and Hart, 

1987). 


Shifting visual attention from one eye to the other (without closing the unattended eye) is 
difficult to learn, mentally demanding, and visually fatiguing. Operational experience does not 
appear to minimize the problem; rather, pilots learn how to minimize its impact on their operational 
performance. It is not clear whether specific training programs, developed to aid pilots in 
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developing visual-attention-management skills, would be effective in improving pilot's perfor- 
mance and in reducing visual fatigue. 


Depth Perception 

Because information is presented monocularly, all stereoscopic depth cues for objects in the 
immediate environment are lost. Additionally, the difference between the apparent size and loca- 
tion of objects viewed direcdy or through the sensor can provide conflicting information about the 
distance of objects in the environment (Roscoe, 1987). Although binocular systems have been 
proposed by government and industry researchers, the technical problems associated with fusing 
information from two sensors to provide a natural binocular image have not been solved adequately 
for operational use. Alternatively, the same image could be presented to both eyes-a biocular dis- 
play. While this would eliminate the problem of binocular rivalry, it would limit pilots’ abilities to 
gain peripheral cues outside the cockpit, see instruments inside the cockpit, or maintain at least one 
dark-adapted eye. And, it would still not provide stereoscopic information. 


Display Magnification 

The displayed information is collimated to optical infinity and magnified to represent a 1 : 1 
mapping with respect to the environment. However, the apparent magnification is not perceived as 
being 1:1. This creates a problem when precise distance judgments must be made, as during 
landing or formation flying. Pilots report that objects appear to be closer when viewed through a 
FLIR than they would with the unaided eye, particularly when the FLIR image is very bright 
(Bennett and Hart, 1987). Other distance misperceptions may also result from the difference in 
light and dark adaptation of the aided and unaided eyes (the Pulfrich effect, see Tyler, 1974) and 
from misaccommodation of the eyes (Roscoe, 1985). Pilots have reported that they minimize this 
problem by confirming range with their left eye. This forces them to shift their visual attention 
back and forth between the aided (light-adapted) and unaided (dark-adapted) eyes (Bennett and 
Hart, 1987). 


Summary 

Current technology systems provide pilots with a wealth of information that would not 
otherwise be available at night or in low visibility. Without visual aiding, the range of environ- 
ments in which low-level missions could be performed would be severely reduced. However, 
many properties of existing systems (e.g., low resolution; the restricted scale of gray shades; and a 
limited, monocular field of view) contribute to the creation of images which contain only a small 
part of the information that is available through direct vision in good visibility. Thus, pilots are 
deprived of essential information about small obstacles or targets and the detail required to identify 
larger objects. The adverse effects of degraded image quality may impose significant workload 
and visual fatigue. However, the effects of these factors seem to be relatively unequivocal and 
predictable, in comparison with the effects of sensor location, binocular rivalry, and depth percep- 
tion. These phenomena may appear in different forms during different flight maneuvers and for 
different pilots. Some individuals may even experience exactly the opposite phenomena than 
others experience. For example, some pilots tend to overestimate, while others underestimate, size 
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and distances. Thus, considerable skill and experience is required for NOE flight with the PNVS, 
and even highly trained pilots consider it to be a highly demanding task. 


ISSUES RELATED TO HELICOPTER CONTROL 


In addition to all of the human factors problems related to the nature of the thermal image 
and to the design of the PNVS/IHADSS, one has to bear in mind that the system is installed on a 
moving six-degree-of-freedom platform which is designed to perform a variety of demanding 
operational tasks. Some of the most difficult tasks involve NOE flight, off-axis tracking, and 

hovering. 

To perform each of these tasks well, pilots must learn to distinguish the effects of conffol 
inputs (e.g., changes in the direction, speed, or orientation of the helicopter itself) from the effects 
on the visual display of changes in sensor orientation induced by the pilot s head movement. Dis- 
orientation can result from a conflict between vestibular cues (based on vehicle motion) and visual 
cues (obtained through the sensor). Pilots learn to limit their head movements (to reduce vertigo) 
and to time them to achieve a stable direction of gaze before changing their direction of flight (to 
reduce spatial disorientation.) They must balance this requirement for limited head movement 
against their need to scan the environment (to obtain an acceptable field of regard or to track mov- 
ing targets) to compensate for the sensor's narrow FOV. 


NOE Flight 

In NOE flight, pilots must fly at very low altitudes among natural and human-made terrain 
features. Even in good visibility, this presents a challenging task for which there is a very low tol- 
erance for error. In reduced visibility, the requirement to perform the same mission using visual 
aids (such as the FLIR/PNVS) is even more difficult. In NOE flight, problems associated with the 
quality of the visual display, the absence of stereoscopic depth cues, display magnification, and the 
offset location of the sensor are particularly pronounced and combine to make rapid and accurate 
range estimates, required to avoid natural and human-made obstacles, very difficult. In addition, it 
is difficult for pilots to maintain a sense of their general geographical orientation because of the 
narrow FOV of the sensor and limitations in its range; their view of the world through which they 
are flying is effectively limited to nearby terrain features. Also, the degraded and dynamically 
changing quality of the visual representation of objects in the environment make it difficult for 
pilots to detect and recognize otherwise familiar objects and terrain features. Finally, the narrow 
FOV of the sensor and limitations in the display of surface texture inhibit pilots' abilities to main- 
tain visual control of speed, heading, and altitude. 

These limitations combine to create a flight environment where pilots must fly slower and 
higher to maintain acceptable margins for safety. Further, performing this task imposes high 
visual and cognitive demands on pilots and is very fatiguing, thereby limiting the duration of mis- 
sions and flight hours. 
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Hovering 


In an inherently unstable vehicle, or without stability and control augmentation systems, 
hovering is extremely difficult and performance is worse when visual information is obtained 
through a helmet-mounted display (Landis & Aiken, 1982). Even in a relatively stable vehicle, 
such as the Apache, visual reference points vary whenever the pilot moves his or her head and 
depth cues are difficult to obtain from the monocular display. Because display resolution is lim- 
ited, subde relative motion cues may be difficult to detect. In addition, peripheral visual cues that 
provide an important source of motion information with direct vision are limited on the PNVS/ 
IHADSS. Thus, pilots supplement the sensor imagery with information available to the unaided 
eye (to provide the necessary peripheral motion cues) and with information provided by super- 
imposed symbology. 


Off-axis Tracking 

Since the sensor is attached to the helicopter, its orientation and position with respect to the 
environment reflect the forward, lateral, and vertical translation and pitch, roll, and yaw of the 
vehicle. However, within the boundaries of its range of movement, the azimuth and elevation of 
the FLIR sensor is independent of the orientation of the helicopter. Spatial disorientation and 
reduced flightpath control performance may occur when pilots look in a different direction than the 
vehicle is moving ("off-axis" tracking). Visual motion cues relevant for flightpath control are more 
difficult to interpret when they are obtained through a sensor that is oriented off-axis to the direc- 
tion of flight (see fig. 6). Peripheral cues (which could integrate the conflicting sources of infor- 
mation) are limited by the narrow FOV, thereby intensifying the problem. 

Pilots appear to trade off flight-control performance for visual tracking performance; visual 
tracking performance is degraded when it is coupled with the requirement to control the vehicle. In 
addition, visual tracking of curved vehicle trajectories is degraded (in comparison to straight tra- 
jectories) and tracking error is increased as the apparent rate of movement of a target across the 
pilot's visual field is increased (by changes in the distance of a target, the rate of movement of the 
target, and/or the velocity of the pilot's vehicle) (Bennett et al., this volume). 

Pilots report (Bennett and Hart, 1987) that they are able to perform off-axis tracking for 
only short periods of time (no more than a few seconds, depending on the flight mode) before they 
must return the orientation of the sensor to correspond to the direction of flight. Thus, pilots come 
to a hover (when they must visually track a moving target) or they hand a target off to the copilot. 
Research is under way at NASA Ames Research Center (Bennett et al., this volume) and else- 
where, to quantify the range of human performance limitations in performing off-axis tracking and 
to develop display augmentations to improve pilots' performance capabilities. 


Superimposed Symbology 

Several sources of information are often combined on helmet-mounted displays. In the 
Apache AH-64 and the Cobra, computer-generated symbology depicting flight-control information 
is superimposed on the sensor imagery and presented on the HDU. This composite display 
reduces the need for pilots to look at cockpit instruments during low-level flight. 


13-10 



Flight-control Symbology- In the Apache, computer-generated graphic and symbolic 
information about the vehicle's flight and performance status is provided to improve > 
to perform flightpath control. The computer- generated display is visible on the monocleno matter 
where the pilot’s head points. However, since the symbology is always oriented i n the < Section of 
flight, as it would be in a head-up or panel-mounted display, it may not present the flight-contro 
symbology in an orientation that is compatible with the direction the pilot is looking (fig. ). 

Ud to 14 flight parameters may be displayed to ensure vertical and horizontal orientation. 
Different subsets of information are presented for different mission segments (e g., hover, transi- 
tion to hover) (fig. 8). The sensitivity of some elements of the display changes for different t 
(e.g., sensitivity is increased during hover and for given altitudes). Although such increased sen- 
sitivity is essential to allow pilots' to maintain a stable hover, learning how to interpret vanatio 
the movement of symbolic display indicators is difficult during initial training (Bennett and Hart, 

1987). 

HDU displays of flight symbology are extremely useful, particularly in NOE flight when 
pilots are too busy to look at cockpit instruments. However, perceptual problems maybe crea e 
by the interference between the computer- generated symbology (which is always onented 
direction the vehicle is moving) and the video display upon which it is superimposed (whi<^ 
oriented in the direction the pilot is looking) (see fig. 8 ). Furthermore, movement of the HDU 
symbology may induce a perception of apparent motion in the video display. 

Pilots learn to ignore the superimposed indicators (when they do not need the information) 
to resolve the problem of display clutter. This is analogous to ignoring the dividers between panes 
of glass in a multipane window when looking outside-one only ’ sees the outside scene. How- 
ever for windows, there is a difference in accommodation between the two sources of informa- 
tion facilitating a difference in attentional focus. For the PNVS/IHADSS, on the other hand, the 
optical distance of both visual display elements is the same, increasing the difficulty that P^o^have 
in focusing on one source of visual information or the other. Pilots report that they tend to look 
through the symbology at the outside scene (at the expense of viewing critical flight data) or vi 
versa (Bennett and Hart, 1987). When they feel that they do need the information, howeyer. they ^ 
include it in their scan. One symbol that remains essential is the diamond that represents the 
ofdie helicopter. It was added at the request of the first pilots to fly the PNVS to ortent them to 
their direction of flight regardless of where the sensor was pointing. 

Targeting Information- Weapons selection, aiming, and other targeting mfonnation can 
be superimposed on a display, as well. The Target Acquisition/Designation s Y stem CTAOS) in the 
AH-64 provides FLIR, direct-vision optics, and daylight television display options boresighted to 
a common line of sight. The TADS has narrow and wide FOV alternatives and an electronic 
"zoom” capability. In the current configuration, the TADS is used by the GQpiiot/gunner. _ ow- 
ever in the environment envisioned for more advanced helicopters, such as the LHX, a single pilot 
might be required to use a helmet-mounted PNVS for both primary vehicle control and for 
weapons delivery. The visual display might be provided by one sensor or a fused combination of 
different sensors. In this situation, it is possible that a pilot might need to look m one direction to 

maintain vehicle control and in another to track, acquire, and fire at enemy targets. Command 

information might be displayed to tell pilots where to look if an automatic target recognition system 
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identified a target in a different direction than they were looking. This could result in a visual 
display of superimposed visual information from three different spatial orientations: (1) computer- 
generated symbology oriented in the direction of flight, (2) the display of FLIR information ori- 
ented in the direction of the pilot's head, and (3) targeting information. 


Effects of Vibration 

Normally, the human eye is stabilized so as to maintain visual fixation in moving environ- 
ments. The vestibular-ocular reflex induces eye movements that oppose those of the head to 
maintain a stationary point of regard during voluntary head movements. In vibrating environ- 
ments, however, the eye may not be capable of compensating for the high-frequency components. 
The detrimental effects of vibration on visual acuity have been well documented (e.g., Griffin, 
1977), particularly for panel-mounted displays, where some of the effects of vibration on instru- 
ment reading can be compensated for by presenting sufficiently large characters and symbols. 

The effects of vibration can be even more severe with helmet-mounted displays, although 
the range of vibrations in advanced-technology helicopters has been reduced considerably. The 
sensor, which is slaved to the pilot's head movement, cannot discriminate involuntary, vibration- 
induced helmet movements from those initiated by the pilot. Relative motion is created between the 
image on the head-coupled display and the eye, resulting in retinal blurring, increased errors, and 
longer responses. It has been suggested that such "involuntary" head movements might be sensed 
by an onboard computer and that this information could be used to provide a stabilized display for 
the pilot (Velger, Grunwald & Merhav, 1986). Based on a computer simulation of the vibration 
frequencies of helicopters, an adaptive noise-canceling technique has been developed that mini- 
mizes the relative motion between viewed images and the eye by shifting displayed images in the 
same direction and magnitude as the induced reflexive eye movements. The filter stabilizes the 
images in space while still allowing low-frequency, voluntary head motions required for aiming 
accuracy. 


The Helmet 

The IHADSS apparatus is relatively heavy (4 lb), producing discomfort and fatigue. And 
most of the weight is in front; counter- balancing weights do not completely eliminate the muscle 
fatigue induced by maintaining heads-up attention to the visual scene. In addition, to reduce the 
problems associated with involuntary head motion within the helmet, a snug fit is essential, which 
may produce "hot spots," further increasing discomfort. However, the pilots’ helmets rarely fit 
perfectly with the consequence that the position of the monocle, which is attached to the helmet, 
may shift in flight. Furthermore, pilots' head movements within an imperfectly fit helmet may not 
be directly translated into helmet movements (which actually control the orientation of the sensor), 
although this does not present a major problem with current systems. 


Grew Size 

All contemporary military helicopters have a flight crew of at least two. In attack heli- 
copters one crew member is primarily responsible for flying the vehicle, while the other is respon- 
sible for navigation, target selection, and weapon control. Recently, the U.S. Army considered the 
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possibility of fielding a single-pilot helicopter. If a single pilot was required to perform a typical 
Apache mission, he or she would have to simultaneously control the helicopter during demanding 
flight maneuvers (e.g., NOE, hover) while detecting, acquiring, and destroying targets. It is well 
established in the motor-control literature that the concurrent performance of any two nonsynchro- 
nized motor tasks is extremely demanding and very difficult (e.g., Keele, 1986). Thus, effective 
off-axis target tracking seems to be feasible only if manual flightpath control demands are low (as 
in high-altitude, straight- and-level flight) or if at least one of the tasks can be automated. Since the 
high-threat battlefield environment requires NOE flight, automated flight and hovering systems 
may be required to effectively release a single pilot from the control of the platform (to enable the 
pilot to accomplish the weapons delivery task), or effective automated target recognition/ 
acquisition systems will be required to provide the pilot with reserve capacity to perform manual 
flightpath control. The successful design of a single-pilot, multipurpose helicopter will rely on the 
accumulation of a considerable body of human factors data in the areas of human information pro- 
cessing, workload, motor control, perception, and skill acquisition. 


Summary 

Helmet-mounted pilot night- vision systems do what they are intended to do. They allow 
pilots to perform NOE missions at night and under low-visibility conditions. They do so at a con- 
siderable cost to the pilots, however, and adequate training can provide only a partial solution. 

Current PNVS/IHADSS systems provide pilots with a monocular display of monochrome 
video images with limited resolution. The detector is not sensitive to natural variations in shading 
in the terrain and provides a narrow FOV from a displaced visual eyepoint. The appearance of 
thermal images may deviate substantially from optical images, and it changes with environmental 
conditions. The quality of the displayed image is further affected by (1) the existence of thermal 
contrasts in the environment; (2) the number of gray shades with which the sensor represents tem- 
peratures differences; (3) atmospheric conditions; (4) the selected polarity, gain, and level, and 
(5) vibration. Finally, there are additional limitations created by the display system itself (e.g., the 
resolution of the CRT and its monocular format). 

These and other characteristics of current technology systems combine to provide pilots 
with limited visual cues under many circumstances. This, in turn, inhibits their ability to fly as low 
or as quickly as they might with optimal visual information. Some of the specific perceptual and 
cognitive problems that might contribute to such limitations in performance are (1) binocular rivalry 
(due to the monocular mode of presentation); (2) inaccurate range estimation (due to the offset sen- 
sor location); (3) loss of peripheral motion cues (due to the narrow FOV); (4) loss of directional 
orientation during off-axis tracking; (5) difficulty in identifying objects (due to limited display res- 
olution and contrast and the unique properties of thermal images); and (6) loss of geographical ori- 
entation (due to the narrow FOV and limitations in the line of sight created by terrain features that 
obscure forward vision during NOE flight). Fatigue, especially visual fatigue, presents a particu- 
larly severe problem. And all of the issues discussed above may limit pilots' confidence in their 
ability to control their aircraft at low altitudes where misinterpretation of the structure of the terrain 
may have severe consequences. Finally, in addition to the operational limitations reported by 
experienced pilots, significant problems have been reported during training. 

Although alternative designs have been suggested, there is insufficient information about 
human perceptual and performance limitations (and their interactions) to provide significantly 
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improved specifications, training programs, or alternative designs. Additional research is required 
to determine the most critical problem areas and to propose solutions that consider the human as 
well as the development of technology. Even though critical human factors problems with night- 
vision systems have already been identified, relatively little research is currently being conducted. 
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Figure 1.- Diagram of the vertical and horizontal FOV and fields of regard of the FLIR sensor. 


13-16 



ORIGINAL’ PAGE 

BLACK AND WHITE PHOTOGRAPH 



Figure 2.- PNVS helmet-mounted display unit . 
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Figure 3.- Thermal display with superimposed flight-control symbology. 
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Figure 4.- Temperature distributions of different materials during a 24-hr period ("a" indicates the 
occurrence of crossover; "b" the time of day when the temperature differences are greatest) 
(Berry et al., 1984). 
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1 — Digital ground speed 

2 — Digital indicatad airspaad 

3 — Digital torque 

4— Magna tic haading 

5 — Doppler ataer Indicator 

Figure 7.- Stylized example of different spatial orientations for FLIR imagery and superimposed 
computer-generated symbology. 


6— Digital altitude 

7 — Analog altitude 

8— Vertical spaed indicator (VSI) 
9-14— Central synbology 
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Figure 8.- The complete PNVS symbology set. 
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